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the camera position. The virtual endoscopy image and the axial image are updated on the basis of the generated virtual endoscopy 
image data. As a result, the user can easily define and modify the path using the image generated by volume rendering as a reference 
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data. 



WO 02/41767 Al 111 111 II 111 III I HI I II III III 1 1 III I III I II 1 1 II I III I III 1 1 II II I II 1 1 



GB, GR, IE, IT, LU, MC, NL, PT, SE, TR), OAPI patent 
(BF, BJ, CF, CG, CI, CM, GA, GN, GQ, GW, ML, MR, 
NE, SN, TD, TG). 

Published: 

— with international search report 



For two-letter codes and other abbreviations, refer to the "Guid- 
ance Notes on Codes and Abbreviations" appearing at the begin- 
ning of each regular issue of the PCT Gazette. 



WO 02/41767 



PCT/KR01/02017 



System and Method for Displaying a Virtual Endoscopy and Computer 
Readable Medium Storing thereof 

BACKGROUND OF THE INVENTION 

(a) Field of the Invention 

The present invention relates to a virtual endoscopy system and 
method, and a computer readable recording medium for storing the method. 
More specifically, the present invention relates to a virtual endoscopy 
system and method, which are capable of setting a path using a cross- 
sectional image and a volume rendering image in a virtual endoscope, and 
a computer readable recording medium for storing the method. 

(b) Description of the Related Art 

In general, an image reconstruction system including a virtual 
endoscopy system has various application fields. In particular, the image 
reconstruction system can be used in the medical image field for a three- 
dimensional (3D) human body. 

In particular, a 3D medical imaging technique involves generating a 
three-dimensional image from a series of axial images obtained from 
Computed Tomography (CT) or Magnetic Resonance Imaging (MR1). Only 
using 2D axial images makes it difficult to imagine the 3D shape and to 
reconstruct the arbitrary section, which are very useful for diagnosis. When 
the 3D medical imaging technique is used, it is possible to find a particular 
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position and to plan an operation method with the better sense of reality. 

Conventional virtual endoscopy systems use axial images in order 
to control the movement of a virtual camera. However, it is difficult to 
determine the corresponding camera position for doctors to be observed by 
5 using axial images. Also, the systems cannot easily perform automatic 
navigation. 

The improved technique based on the automatic segmentation and 
surface rendering has been developed. However, automatic segmentation 
cannot be easily performed especially in a complicated structure. In 
10 addition, automatic path generation algorithm is not so perfect to find a 
path that a user wants to find. 



SUMMARY OF THE INVENTION 

It is an object of the present invention to provide a virtual 
15 endoscopy system to set a path on a three-dimensional (3D) volume 
rendering image without preprocessing, which can reduce time required for 
diagnoses. And by displaying a multi-planar reformation image including 
the path,the efficiency of diagnoses can be improved. 

It is another object of the present invention to provide a virtual 
20 endoscopy method using the virtual endoscopy system. 

It is another object of the present invention to provide a computer 
readable recording medium for storing a program including the virtual 
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endoscopy method. 

In one aspect of the present invention, there is provided a virtual 
endoscopy system, comprising an input/storage unit for reading or 
transferring the 3-dimensional volume data from source devices into the 
target devices for further processing; an endoscopy image processing unit 
for analyzing and displaying spatial distribution of the characteristics to a 
3D image on the basis of data stored in the input/storage unit, processing 
the 3D image so as to display it as a reference image, generating an 
endoscopy image along a predetermined path directly displayed on the 
reference image, and processing the endoscopy image so as to display it; 
a display for displaying the reference image and the endoscopy image 
along the path; and an input unit for letting a user designate the path from 
the reference image. 

The endoscopy image processing unit comprises a reference 
image processor for generating the 3D reference image so as to display it 
from the volume data stored in the input/storage unit, receiving a 
predetermined path that passes through an inner structure whose 
endoscopy image is to be seen through the input methods in the form of 
predetermined curved line on the reference image, and processing the 
curved data; a path converter for determining the position of a voxel 
corresponding to the wall of the inner structure whose endoscopy image is 
to be seen in the volume data from the path input to the reference image 
processor, and correcting a path so that the path can actually pass through 
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the inner structure; and a multi-planar reformation image composer for 
generating a multi-planar reformation image including the path along the 
corrected path obtained by the path converter. 

In another aspect of the present invention, there is provided a 
virtual endoscopy method for storing primitive image data, classifying the 
density value of the stored primitive image data, and displaying a virtual 
endoscope, comprising (a) generating a volume rendering image 
corresponding to a part to be observed from the 3D volume data; (b) 
displaying the generated volume rendering image, an axial image, a virtual 
endoscopy image, and a path MPR image; (c) updating display of the path 
MPR image as a path is set in either the axial image or a 3D volume 
rendering image according to the manipulation of a user; (d) checking 
whether the position of a camera is requested to be changed according to 
the manipulation of the user; and (e) generating virtual endoscopy image 
data according to the change of the camera position and updating the 
virtual endoscopy image and the axial image on the basis of the generated 
virtual endoscopy image, when the position of the camera is requested to 
be changed. 

The camera displays the volume rendering image, the axial image, 
the virtual endoscopy image, and the path MPR image using a 
predetermined viewing vector; updates the volume rendering image, the 
axial image, the virtual endoscopy image, and the path MPR image using a 
changed viewing vector obtained by controlling the viewing vector; and 
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obtains a 3D image effect on the axial image. 

In another aspect of the present invention, there is provided a 
computer readable recording media for storing a program including a 
method for displaying a virtual endoscope by storing primitive image data 

5 and classifying the density value of the stored primitive image data, 
comprising (a) generating a volume rendering image corresponding to a 
part to be observed from the 3D volume data; (b) displaying the generated 
volume rendering image, an axial image, a virtual endoscopy image, and a 
path MPR image; (c) updating display of the path MPR image as a path is 

10 set in either the axial image or a 3D volume rendering image according to 
the manipulation of a user; (d) checking whether the position of a camera is 
requested to be changed according to the manipulation of the user; and (e) 
generating virtual endoscopy image data according to the change of the 
camera position and updating the virtual endoscopy image and the axial 

15 image on the basis of the generated virtual endoscopy image, when the 
position of the camera is requested to be changed. 

According to the virtual endoscopy system and method, and a 
computer readable recording medium for storing the method, a user can 
easily define and modify a path and control the direction of the camera 

20 from two dimensions to three dimensions using ah image generated by 
volume rendering without preprocessing with respect to the 3D volume 
data as a reference image. 



5 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The accompanying drawings, which are incorporated in and 
constitute a part of the specification, illustrate an embodiment of the 
invention, and, together with the description, serve to explain the principles 
5 of the invention: 

FIG. 1 shows a virtual endoscopy system according to an 
embodiment of the present invention; 

FIG. 2 shows an example of a screen of a virtual endoscopy 
system according to the present invention; 
io FIG. 3 is a flowchart for explaining a virtual endoscopy method 

according to the present invention; 

FIG. 4 is a flowchart for explaining S400 of FIG. 3 in more detail; 

FIG. 5 is a flowchart for explaining S420 of FIG. 4 in more detail 

FIG. 6 explains processes of searching for the position of an actual 
15 inner organ in three dimensions from a point input to a reference image in 
the virtual endoscopy system according to the present invention; 

FIG. 7 explains the generation of a cross-sectional reformation 
image including a path according to the present invention 

FIG. 8 explains the generation of a cross-sectional reformation 
20 image parallel to or vertical to the path according to the present invention; 

FIG. 9 explains the three-dimensional controlling of a camera in 
two dimensions according to the present invention; 

FIG. 10 shows a three-dimensional (3D) volume rendering image 
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used as the reference image of the virtual endoscopy system according to 
the present invention; 

FIG. 11 shows an example of loading a screen provided by a 
preferred virtual endoscopy system according to the present invention on 
5 the basis of the 3D volume rendering image of FIG. 1 0; and 

FIG. 12 shows a screen providing a function of arbitrarily modifying 
the path once designated by a user in FIG. 11 in the preferred virtual 
endoscopy system according to the present invention. 

10 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

In the following detailed description, only the preferred embodiment 
of the invention has been shown and described, simply by way of 
illustration of the best mode contemplated by the inventor(s) of carrying out 
the invention. As will be realized, the invention is capable of modification in 
various obvious respects, all without departing from the invention. 
Accordingly, the drawings and description are to be regarded as illustrative 
in nature, and not restrictive. 

FIG. 1 shows a virtual endoscopy system according to an 
embodiment of the present invention. 

Referring to FIG. 1, a virtual endoscopy system according to the 
present invention includes an input/storage unit 10, an endoscopy image 
processing unit 20, a display 30, and an input unit 40. 



15 



20 
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The input/storage unit 10 reads or transfers the 3-dimensionaI 
volume data from source devices into the target devices for further 
processing. For example, in the case of receiving and storing the volume 
data as a set of results from computed tomography (CT) or magnetic 

5 resonance imaging (MRI) section scanning, the input storage unit can be 
formed of access hardware for receiving data from the respective 
measurement apparatuses and a hard disk or a memory device for storing 
the data. In the case of storing the measured result in another storage 
medium and reading the measured results or storing the measured results 

10 in a storage medium such as a hard disk, the input storage unit 10 can be 
formed of access hardware for reading data from storage apparatuses 
inside/outside the three-dimensional image processing system and a hard 
disk or a memory device for storing the data. 

The endoscopy image processing unit 20 includes a reference 

15 image processor 22, a path converter 24, and a multi-planar reformation 
image composer 26. The endoscopy image processing unit 20 analyzes 
and displays the spatial distribution of the characteristics as a 3D image on 
the basis of the data stored in the input/storage unit 10, processes the 3D 
image so as to display it as a reference image, generates an endoscopy 

20 image along a predetermined path directly displayed on the reference 
image, and processes the endoscopy image to be displayed. 

More specifically, the reference image processor 22 displays the 
3D reference image from the volume data stored in the input/storage unit 
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10, receives a predetermined path passing through an inner structure 
whose endoscopy image is desired to be seen through the input unit 40 in 
the form of predetermined curved data on the reference image, and 
processes the curved data. 
5 The path converter 24 determines the position in the volume data 

of a voxel corresponding to the wall of the inner structure whose 
endoscopy image is to be seen from the path input to the reference image 
processor 22, and corrects the path so that the inner structure can actually 
pass through the path due to the position of the voxel. 
10 The multi-planar reformation image composer 26 generates a 

multi-planar reformation image including the path according to the 
corrected path obtained by the path converter 24. 

The endoscopy image processing unit 20 is a means for calculating 
and processing the volume data. The endoscopy image processing unit 20 
15 can be realized by a computer that can perform a series of calculation 
processes, a set of computers in which data can be exchanged between 
computers, and an application-specific integrated circuit (ASIC). 

The display 30 displays the reference image and displays the 
endoscopy image along the path. The input unit 40 lets a user designate 
20 the path from the reference image. 

FIG. 2 shows an example of a screen of the virtual endoscopy 
system according to the present invention. 

Referring to FIG. 2, the screen of the virtual endoscopy system 
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according to the present invention includes the four viewers of a 3D volume 
rendering image 100, an axial image 200, a virtual endoscopy image 300, 
and a path MPR image 400. Volume rendering refers to the work of 
extracting visual image information required for various applications from 
5 volume data of a 3D space. 

When a camera is set by the manipulation of a user in the 3D 
volume rendering image 100 of a displayed screen, the axial image 200 
and the virtual endoscopy image 300 are generated, to thus update a 
previously displayed image. Also, as a path is set by the manipulation of 
10 the user, the path MPR image 400 is generated in connection with the path, 
to thus update the previously displayed image. 

As the camera is set by the. manipulation of the user in the 
displayed axial image 200, the path MPR image 400 is generated, to thus 
update the previously displayed image. As the path is set by the 
15 manipulation of the user, the virtual endoscopy image 300 is generated, to 
thus update the previously displayed image. 

Operations for displaying the various images will now be described 
in more detail. 

FIG. 3 is a flowchart for explaining a virtual endoscopy method 
20 according to the present invention. 

Referring to FIGs. 2 and 3, the scan data of the human body 
structure obtained through apparatuses such as the CT and the MRI, that 
is, primitive image data, is stored (S100). 
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The 3D volume rendering image 100 is generated so that an 
observed part can be clearly shown by a method of classifying a density 
value in real time and deleting unnecessary parts in order to look at the 
structure to.be observed according to the manipulation of the user (S200). 

The 3D volume rendering image 100, the axial image 200, the 
virtual endoscopy image 300, and the path MPR image are displayed on 
the four viewers shown in FIG. 2 (S300). 

A corresponding path is set as a path set request, and is input 
according to the manipulation of the user on the axial image or the 3D 
volume rendering image (S400). A path can be set or changed on the axial 
image. However, it is preferred that a user directly sets a path on the 3D 
volume rendering image in the form of an arbitrary straight or curved line 
for the convenience of the user. 

An image is generated along the path set by mapping an axial 
point designated by the 3D volume rendering image 100 to the central point 
of the inner structure to be seen, that is, to a 3D point, and is displayed 
(S500). The path MPR image 400 shown in FIG. 2 will be preferably 
updated. 

It is checked whether the camera is controlled by the manipulation 
of the user (S600). When it is determined that the camera is not controlled, 
the process returns to S500 and an image is continuously displayed. When 
it is determined that the camera is controlled, the virtual endoscopy image 
300 controlled by the camera is generated (S700). At this time, the 

li 
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controlled camera is a kind of a virtual camera that moves on the 
endoscopy image, that is, the point of view of the eyes of the user. It is 
possible to freely control the view angle of the camera. The volume 
rendering image is reformed from the primitive image data in connection 
5 with the freely controlled view angle of the camera, and is displayed. 

The axial image 200 and the virtual endoscopy image 300 are 
updated and displayed (S800). 

It is checked whether the 3D volume rendering image 100 is re- 
manipulated (S900). When it is determined that the 3D volume rendering 
10 image is re-manipulated, the process returns to S200. 

A path-setting method that is a characteristic of the present 
invention will now be described in detail with reference to the attached 
drawings. 

FIG. 4 is a flowchart for explaining S400 of FIG. 3 in more detail. 

15 Referring to FIG. 4, as points are designated by the manipulation of 

the user on the axial image or the 3D volume rendering image for the 
reformation of the path (S410), curved control points are added (S420). 

A path is generated using the formula of a 3D curved line (S430). 
Corresponding points are stored in a predetermined path material structure 

20 along a corresponding curved line at predetermined intervals, for example, 
at each one voxel interval (S440). The various points stored in the path 
material structure will be used as data corresponding to the reset path, in 
response to which path MPR images will be updated and displayed in S500. 
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In S410, when the user uses the axial image 200 shown in FIG. 2 
as the reference image in order to obtain the virtual endoscopy image or 
the path MPR image, it is possible to know the voxel where the axial image 
exists together with the (x, y) data of the displayed reference image, that is, 
5 the axial image. Therefore, it is possible to know the z coordinate value. 
Accordingly, it is possible to know the 3D coordinate (x, y, z), which is the 
position of the axial image on the entire volume data. 

However, when the user uses the 3D image as the reference 
image in order to obtain the virtual endoscopy image or the path MPR 
10 image, the z value must be efficiently recognized. 

A method for recognizing the z value when the user uses the 3D 
image as the reference image, which is another characteristic of the 
present invention, will now be described in more detail. 

FIG. 5 is a flowchart for explaining S420 of FIG. 4 in more detail. In 
15 particular, processes of mapping the axial point designated in the 3D 
volume rendering image to the central point of the structure will be 
described. FIG. 6 explains processes of searching for the position of an 
actual inner organ in three dimensions from a point input to a reference 
image in the virtual endoscopy system according to the present invention. 
20 Referring to FIGs. 5 and 6, as a mouse is input to the axial image 

displayed by the 3D volume rendering according to the manipulation of the 
user (S4210), the minimum value and the maximum value of a depth to be 
examined are set (S4220). 
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The point of a plane coordinate system is converted into the point 
of the spatial coordinate system by multiplying time inverse matrices by 
each other (S4230). 

A predetermined ray is casted from the center of projection (S4240). 
5 The casted ray is a virtual straight ray line from a virtual 3D space rather 
than physicalray. 

It is checked whether the ray encounters the voxel corresponding 
to the wall of the structure (S4250). When it is determined that the ray 
encounters the voxel corresponding to the wall of the structure, the position 
10 where the ray encounters the voxel is recorded in inVolumePoint (S4260). 
The values of the respective unit parts that form the voxel on a straight line, 
that is, volume data and peripheral values, are checked along a given 
viewing vector or in the direction of a camera in the axial image input to the 
screen. Accordingly, it is checked whether the values of the respective 
is units parts and the peripheral values correspond to the density value of the 
wall of the structure. 

It is checked whether the ray encounters the voxel corresponding 
to the wall of the structure (S4270). When it is determined that the ray 
encounters the voxel corresponding to the wall of the structure, the position 
20 where the ray encounters the wall of the structure is recorded in 
outVolumePoint (S4280). 

The result of a middle point between the inVolumePoint and the 
outVolumePoint is output. It is possible to convert the point input from the 
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volume rendering image into the center of the structure on the basis of the 
result (S4290). 

As mentioned above, in FIG. 5, a black voxel has the density value 
of a human body. A first point in the direction where an arrow moves 
5 forward shows a state before entering the inside of the structure. A second 
point shows the inside of the structure. A third point shows a state right 
after passing through the inside of the structure. 

When endoscopy diagnoses are performed by the virtual 
endoscopy system, it is very useful if a user has information on peripheral 
10 structures. In particular, medical staffs want a reformation image including 
the path passing through the central line of the structure. 

In response to such a request of the medical staffs, according to 
the virtual endoscopy system of the present invention, the reformation 
image including the path is shown as the reference image. Also, the image 
15 is created to rotate 360°. Therefore, it is possible to obtain correct 
information on peripheral structures. 

FIG. 7 explains the generation of the cross-sectional reformation 
image including the path according to the present invention. FIG. 8 
explains the generation of a cross-sectional reformation image parallel to or 
20 vertical to the path according to the present invention. 

Referring to FIGs. 7 and 8, a multi-planar reformation image 
including a straight or curved path can be formed by generating scan lines 
one by one in a uniform direction set by the user in the respective points on 
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the path according to the manipulation of the mouse by the user. That is, 
when the cross-sectional image is displayed on a 3D image, the multi- 
planar reformation image cut through a curved line on the 3D image having 
various forms as well as an image cut by a straight line can be generated 
5 and displayed. Because the path MPR image formed at this time displays 
information on the peripheral structures centering on the path passing 
through the central line of the structure, the path MPR image is very useful 
to the medical teams when the endoscopy diagnoses are performed. 

According to the present invention, a curved planar reformation is 

10 performed in a direction where the 3D image is observed in forming the 
multi-planar reformation image using a virtual endoscopy path. Accordingly, 
it is possible to spread the overall length of the anatomical structure 
desired to be seen. Also, it is possible to rotate a curved planar reformation 
image using the virtual endoscopy path as an actual axis by changing the 

is curved planar reformation image according to the rotation of the 3D image. 

As shown in FIG. 8, in the case of reforming a cross-sectional 
plane of a straight line interface method, which is parallel to the virtual 
endoscopy path in the point of view of the virtual endoscope, it is possible 
to freely rotate the cross-sectional plane using the direction of the virtual 

20 endoscopy path as an axis. The reformation of the cross-sectional plane of 
the straight line interface method, which is vertical to the virtual endoscopy 
path in the point of the virtual endoscope, can be realized so as to 
continuously show the vertical cross-sectional plane of the structure to be 



WO 02/41767 



PCT/KR01/02017 



observed, while moving according to the movement of the point of view of 
the virtual endoscope. 

That is, it is possible to easily understand the view of the virtual 
endoscopy image through the multi-planar reformation image by selecting 
5 and using an appropriate multi-planar reformation method according to the 
characteristics of the structure to be observed in the point of view of the 
virtual endoscope. 

A method of spreading the overall length of the anatomical 
structure desired to be seen by performing the curved planar reformation in 
10 the direction where the 3D image is observed will be realized as follows. 

Information on a curved line path that follows the center of the 
structure is expressed as points separated from each other by one. The 
points are stored as a 3D coordinate system value (x, y, z). Therefore, a 
curved planar reformation image is obtained following a straight line that 
is passes one point on the path because one point on the path and the 
viewing vector are known. 

A value corresponding to a pixel is obtained, while moving forward 
using a directional vector until getting out of the boundary of the volume 
data starting from each point. One line of a reformation image is obtained 
20 when the above operation is performed with respect to one point. 

Therefore, when the above operation is performed with respect to 
all of the 3D points, a perfect reformation image is formed. FIG. 7 shows 
the processes. 
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As shown in FIG. 8, in order to reform the cross-sectional plane 
parallel to or vertical to the path, a normal vector in a point on the path is 
used. The normal vector value of each point that forms the path is a value 
obtained by differentiating the formula of a curved line. Therefore, in order 

5 to reform the plane parallel to one point on the path, a plane reformation 
image is formed by a plane including a point on the path and a point on the 
normal vector. Because a plurality of planes can be formed, it is preferable 
that a plane is selected by the manipulation of the user. 

Also, one plane is determined because a cross-sectional plane 

10 vertical to one point on the path is vertical to one point and the normal 
vector. Therefore, when a point on the path is selected, it is possible to 
form the plane reformation image by the plane whose normal vector is the 
normal vector of the point. 

A method of obtaining a 3D effect on the axial image, which is 

15 another characteristic of the present invention, will now be described in 
more detail. 

FIG. 9 explains the 3D controlling of a camera in the axial image 
according to the present invention. In particular, the manipulation of the 
direction on the plane and the manipulation of the direction on the depth 
20 will now be described in detail. 

Referring to FIG. 9, the position to be observed can be designated 
by clicking the mouse in the 3D image created by the axial image or the 
volume rendering image. The virtual endoscopy viewing direction can be 
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controlled to the planar direction of the x, y coordinate and the depth 
direction of the z coordinate in the axial image and the 3D image. 

According to the present invention, the user can easily select and 
grasp the position of the virtual endoscope by newly rendering the virtual 
endoscopy image whenever the point of view and the viewing direction are 
changed. Accordingly, it is possible to obtain the 3D image effect in the 
axial image. 

The manipulation method will now be described in the point of view 
of the user who uses the virtual endoscope. 

The position and direction of the current camera are marked with 
red in the volume rendering viewer and the axial image. In order to change 
the position of the camera, for example, the user clicks the left mouse 
button. In order to change the direction of the camera, for example, the 
user clicks the right mouse button. However, when a common axial image 
is used as the reference image of the virtual endoscope, it is difficult to 
display the depth direction on the two dimensions. 

However, according to the virtual endoscopy system of the present 
invention, it is possible to change the direction mode of the z coordinate as 
shown in FIG. 8. That is, a direction conversion mode is changed by a 
toggle method when the user clicks the arrow with the right mouse button. 
In the case of red, planar direction controlling can be performed. In the 
case of blue, depth direction controlling can be performed. In the case of 
red, the direction changes on a plane vertical to the viewing direction of the 
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camera according to the movement of the mouse. In the case of blue, the 
direction changes on a plane including the vector of the viewing direction of 
the camera. A 3D manipulation can be performed on the axial image. 

FIG. 10 shows the 3D volume rendering image used as the 
5 reference image of the virtual endoscopy system according to the present 
invention. In particular, an image obtained by visualizing a bronchus by the 
three-dimensional volume rendering in order to perform an endoscopy on 
the bronchus is shown. 

Referring to FIG. 10, in order to look at the bronchus by the virtual 
10 endoscopy, classification is performed by the density value corresponding 
to the bronchus in the 3D volume rendering, unnecessary parts are deleted 
so that the bronchus is shown, and the viewing direction of the camera or 
the user is controlled. 

FIG. 11 shows an example of loading a screen provided by a 
15 preferred virtual endoscopy system according to the present invention on 
the basis of the 3D volume rendering image of FIG. 10. 

Referring to FIG. 11, in the virtual endoscopy system, the four 
viewers shown in FIG. 2 are shown. The position of the camera is marked 
with red in each viewer. An example of setting a direct path in the 3D 
20 reference image, which is a characteristic of the present invention, is 
shown. 

When the path is set by the manipulation of the user, a multi-planar 
reformation image including the path is generated in a MPR path viewer. 
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The position of the camera is displayed in the multi-planar reformation 
image including the path. In order to change the direction of the multi- 
planar reformation image, including the path, the mouse is used in the 
volume rendering image viewer. The direction of the reformation image is 
5 marked, with a blue line in the 3D volume rendering viewer among the four 
viewers. 

FIG. 12 shows a screen that provides a function of arbitrarily 
modifying the path once designated by the user in FIG. 11 in the preferred 
virtual endoscopy system according to the present invention. 
10 Referring to FIG. 12, processes of directly modifying the path in the 

3D reference image are shown. The path is modified by moving the point 
on the path with the mouse. 

Equipment for measuring a length on the path created as such is 
included in the virtual endoscopy system according to the present invention 
15 for the convenience of the user. 

Various changes in form and details may be made in the virtual 
endoscopy system, the virtual endoscopy method using the same, and the 
computer readable recording media for storing the method, without 
departing from the spirit and scope of the invention. The virtual endoscopy 
20 system, the virtual endoscopy method using the same, and the computer 
readable recording media for storing the method are not limited to the 
preferred embodiment. 

For example, a light pen, the keyboard, and various input 
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apparatuses as well as the mouse can be used as the input means. The 
virtual endoscopy system, the virtual endoscopy method using the same, 
and the computer readable recording media for storing the method can be 
widely applied to the designing and constructing of 3D structures such as 
5 automobiles, ships, and buildings, as well as systems for processing 
medical images. 

As mentioned above, according to the virtual endoscopy system 
according to the present invention, the user can easily define and modify 
the path using the image generated by the volume rendering as the 

10 reference image without preprocessing on the 3D volume data. The MPR 
image is generated along the set path. Accordingly, it is possible to more 
easily understand the circumstances around the structure. 

The above-mentioned virtual endoscopy that is an integrated 
environment of various kinds of medical images is obtained by generalizing 

15 high technologies. According to the present invention, the virtual 
endoscopy can be developed into an integrated system including various 
images and technologies. 

While this invention has been described in connection with what is 
presently considered to be the most practical and preferred embodiment, it 

20 is to be understood that the invention is not limited to the disclosed 
embodiments, but, on the contrary, is intended to cover various 
modifications and equivalent arrangements included within the spirit and 
scope of the appended claims. 



As mentioned above, because the virtual endoscope according to 
the present invention is not inserted into a patient, it is possible to look at 
the insides of organs without pain. 

Also, according to the prior endoscopy, it is possible to observe the 
insides of the organs only in the direction where the camera moves forward. 
However, according to the virtual endoscopy of the present invention, it is 
possible to observe the insides of the organs in the direction where the 
camera moves backward. Accordingly, the virtual endoscopy according to 
the present invention is very useful for diagnoses. 

The virtual endoscopy system has not been widely used because it 
is difficult to manipulate the virtual endoscopy system due to the 
inefficiency of interface. However, according to the present invention, the 
user can easily manipulate the camera and various reference images can 
be shown. Accordingly, it is possible to improve the efficiency of the virtual 
endoscopy. 
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WHAT IS CLAIMED IS: 

1 . A virtual endoscopy system, comprising: 

an input/storage unit for reading or transferring the 3-dimensional 
volume data from source devices into the target devices for further 
5 processing; 

an endoscopy image processing unit for analyzing and displaying 
spatial distribution of the characteristics to a 3D image on the basis of data 
stored in the input/storage unit, processing the 3D image so as to display it 
as a reference image, generating an endoscopy image along a 
10 predetermined path directly displayed on the reference image, and 
processing the endoscopy image so as to display it;. 

a display for displaying the reference image and the endoscopy 
image along the path; and 

an input unit for letting a user designate the path from the 
15 reference image. 

2. The virtual endoscopy system of claim 1 , wherein the endoscopy 
image processing unit comprises: 

a reference image processor for processing the 3D reference 
20 image so as to display it from the volume data stored in the input/storage 
unit, receiving a predetermined path that passes through an inner structure 
whose endoscopy image is to be seen through the input means in the form 
of predetermined curved data on the reference image, and processing the 
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curved data; 

a path converter for determining the position of a voxel 
corresponding to the wail of the inner structure whose endoscopy image is 
to be seen in the volume data from the path input to the reference image 
processor, and correcting a path so that the path can actually pass through 
the inner structure; and 

a multi-planar reformation image composer for generating a multi- 
planar reformation image including the path along the corrected path 
obtained by the path converter. 

3. A virtual endoscopy method for storing primitive image data, 
classifying the density value of the stored primitive image data, and 
displaying a virtual endoscope, comprising: 

(a) generating a volume rendering image corresponding to a part to 
be observed from the 3D volume data; 

(b) displaying the generated volume rendering image, an axial 
image, a virtual endoscopy image, and a path MPR image; 

(c) updating display of the path MPR image as a path is set in 
either the axial image or a 3D volume rendering image according to the 
manipulation of a user; 

(d) checking whether the camera position is requested to be 
changed according to the manipulation of the user; and 

(e) generating virtual endoscopy image data according to the 
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change of the camera position and updating the virtual endoscopy image 
and the axial image on the basis of the generated virtual endoscopy image, 
when the camera position is requested to be changed. 

5 4. The method of claim 3, further comprising (f) checking whether it 

is requested to reform the volume rendering image according to the 
manipulation of the user, and returning to the step (a) when it is determined 
that it is requested to reform the volume rendering image. 

5. The method of claim 3, wherein the step (c) further comprises: 
(c-1) generating the formula of a 3D curved line by adding the 
control point of the formula of the curved line as a point that is designated 
according to the manipulation of the user in either the axial image or the 3D 
volume rendering image; 

(c-2) generating a curved line path using the formula of the 3D 
curved line; and 

(c-3) storing corresponding points at predetermined intervals along 
the generated curved line path. 

20 6. The method of claim 5, wherein the step (c-2) further comprises 

the step of converting the 3D volume rendering image into the center of the 
structure when the 3D volume rendering image is used as the reference 
image. 
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7. The method of claim 6, wherein the step of converting the 3D 
volume rendering image into the center of the structure comprises: 

setting the minimum value and the maximum value of the depth to 
be examined according to the input of a mouse to the axial image drawn by 
5 volume rendering; 

converting a point of a screen coordinate system into a point in a 
spatial coordinate system by multiplying time inverse matrices by each 
other; 

casting predetermined ray from the center of projection; 
io firstly storing a corresponding position when the casted ray firstly 

encounters a voxel corresponding to the wall of a structure; 

secondly storing a corresponding position when the casted ray 
secondly encounters the voxel corresponding to the wall of the structure; 
and 

15 generating an intermediate point result on the basis of the firstly 

stored position and the secondly stored position. 

8. The method of claim 3, wherein the camera displays the volume 
rendering image, the axial image, the virtual endoscopy image, and the 

20 path MPR image using a predetermined viewing vector; updates the 
volume rendering image, the axial image, the virtual endoscopy image, and 
the path MPR image using a changed viewing vector obtained by 
controlling the viewing vector; and obtains a 3D image effect on the axial 



WO 02/41767 



PCT/KR01/02017 



image. 

9. A computer readable recording media for storing a program 
including a method for displaying a virtual endoscope by storing primitive 
5 image data and classifying the density value of the stored primitive image 
data, comprising: 

(a) generating a volume rendering image corresponding to a part to 
be observed from the 3D volume data; 

(b) displaying the generated volume rendering image, an axial 
10 image, a virtual endoscopy image, and a path MPR; 

(c) updating display of the path MPR image as a path is set in 
either the axial image or a 3D volume rendering image according to the 
manipulation of a user; 

(d) checking whether the camera position is requested to be 
15 changed according to the manipulation of the user; and 

(e) generating virtual endoscopy image data according to the 
change of camera position and updating the virtual endoscopy image and 
the axial image on the basis of the generated virtual endoscopy image, 
when the camera position is requested to be changed. 

20 
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